Multi-video navigation

ABSTRACT

A video tracking system includes a user interface configured to facilitate tracking of a target between video cameras. The user interface includes user controls configured to assist a user in selecting video cameras as the target moves between fields of view of the video cameras. These user controls are automatically associated with specific cameras based on a camera topology relative to a point of view of a camera whose video data is currently being viewed. The video tracking system further includes systems and methods of synchronizing video data generated using the video cameras and of automatically generating a stitched video sequence based on the user selection of video cameras. The target may be tracked in real-time or in previously recorded video and may be track forward or backwards in time.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.14/714,896 entitled “Multi-Video Navigation” filed May 18, 2015, whichis a Continuation of U.S. patent application Ser. No. 13/898,590entitled “Multi-Video Navigation” filed May 17, 2013, which is aContinuation of U.S. patent application Ser. No. 11/726,879 entitled“Multi-Video Navigation” filed Mar. 23, 2007, now U.S. Pat. No.7,777,783, and the complete subject matter of which are incorporatedherein by reference in their entirety.

BACKGROUND

Field of the invention

The invention is in the field of video processing and more specificallyin the field of managing data from multiple video sources.

Related Art

Remote cameras are commonly used to monitor events and provide security.The video output from these remote cameras may be recorded and/ormonitored in real time. When a large number of remote cameras are usedat the same time, it can be difficult to monitor all of the cameras atonce. Further, when a surveillance target moves from the field of viewof one camera to the field of view of another camera, it can bedifficult to follow the target, regardless of whether the images areviewed in real-time or are recorded. For example, it may require hoursto track a person's route through a complicated facility such as anairport. By the time such tracking is complete, the value of theinformation obtained may be significantly reduced, e.g., the target mayhave left the facility or already accomplished some malevolent deed.There is, therefore, a need for improved systems and methods of managingvideo data.

SUMMARY

Various systems and methods of the invention are configured tofacilitate tracking of a target between multiple video sources. Thesevideo sources may be realtime, e.g., live, and/or recorded. Tracking maybe automated, semi-automated, or manual.

Various embodiments of the invention include a user interface configuredfor navigating between video sources, e.g., cameras. This user interfaceis supported by predetermined relationships between the video sources,and by various logic modules as further described herein. The userinterface allows for tracking a target forward in time using live and/orrecorded video. For example, in some embodiments, the user interface canbe used to track a target using recorded video in an accelerated mode.In the accelerated mode, video is processed in a shorter time scale thatit took to record. For example in the accelerated mode, 30 minutes ofvideo may be viewed in at an accelerated rate, e.g., in less than 12minutes, less than 10 minutes, less than 5 minutes, or less than 3minutes. As such, a target can be tracked using recorded video from atime in the past to the present, e.g., until the target is located inreal-time video.

The user interface may also be configured to track a target back in timeusing recorded video. For example, a target first identified inreal-time video may be tracked hack in time to determine where thetarget came from. A target identified in recorded video can be trackedboth forward and backward in time.

During tracking of a target, video segments associated with the targetare optionally stitched together into a sequence of video data. Forexample, as the target moves between video sources, data from thesevideo sources can be spliced together to form a continuous videosequence. As is described further herein, this splicing process mayoccur automatically as a user tracks the target using the userinterface.

Various embodiments of the invention include systems configured toreceive video data from multiple sources, store the received video,display the received video, and process the received video. Thesesystems may include an intermediate layer configured to receivedifferent types of video data and convert the different types of videodata into a default type. These systems may also include video sources,storage media, computing hardware, firmware, software, output devices,and/or the like, configured to process video data and support the userinterface.

Various embodiments of the invention include a system comprising a firstdata storage configured to store a plurality of video data received froma plurality of video sources, respectively, a second data storageconfigured to store data representative of one or more camera topologybetween the plurality of video sources, the one or more camera topologybeing relative to viewpoints of the plurality of video sources, firstlogic configured to generate a user interface for display to a user, theuser interface including a region to display the plurality of video dataand a plurality of user inputs configured for the user to navigatebetween the plurality of video data based on the one or more cameratopology, and second logic configured to select which member of theplurality of video data to display in the user interface responsive tothe plurality of user inputs.

Various embodiments of the invention include a method comprisingidentifying a first member of a plurality of cameras, identifying afirst topological relationship between the first member of the pluralityof camera < >and a second member of the plurality of cameras,identifying a second topological relationship between the first memberof the plurality of cameras and a third member of the plurality ofcameras, the first topological relationship and the second topologicalrelationship being relative to a viewpoint of the first camera,presenting video data generated using the first member of the pluralityof cameras in a user interface, the user interface including a firstuser control and a second user control, associating the second member ofthe plurality of cameras with the first user control based on thetopological relationship between the first member of the plurality ofcameras and the second member of the plurality of cameras, associatingthe third member of the plurality of cameras with the second usercontrol based on the topological relationship between the first memberof the plurality of cameras and the third member of the plurality ofcameras, receiving a user selection of the first user control,presenting video data generated using the second member of the pluralityof cameras in the user interface based on the received user selection,and associating the first user control and the second user control withmembers of the plurality of cameras based on topological relationshipsrelative to a viewpoint of the second member of the plurality ofcameras.

Various embodiments of the invention include a method comprising viewingfirst video data within a user interface, the first video data generatedusing a first member of a plurality of cameras, locating a target withinthe first video data, observing the target leave a field of view of thefirst member of the plurality of cameras in a first direction relativeto a viewpoint of the first member of a plurality of cameras, orobserving the target enter the field of view of the first member of theplurality of cameras from the first direction relative to a viewpoint ofthe first member of the plurality of cameras, selecting a second memberof the plurality of cameras using a first user control associated withthe first direction relative to a viewpoint of the first member of theplurality of cameras, and viewing second video data within the userinterface, the second video data generated using the second member ofthe plurality of cameras. This method may further include observing thetarget leave a field of view of the second member of the plurality ofcameras in a first direction relative to a viewpoint of the secondmember of a plurality of cameras, or observing the target enter thefield of view of the second member of the plurality of cameras from thefirst direction relative to a viewpoint of the second member of aplurality of cameras, selecting a third member of the plurality ofcameras using a user control associated with the first directionrelative to a viewpoint of the second member of a plurality of cameras,and viewing third video data within the user interface, the third videodata generated using the third member of the plurality of cameras.

Various embodiments of the invention include a method comprisingreceiving a user selection of first video data generated using a firstcamera, the first video data including a first time reference, receivinga user selection of second video data generated using a second camera,the second video data including a second time reference, automaticallystitching the first video data to the second video data responsive tothe user selection of the second video data and using the first timereference and the second time reference to form a chronological videosequence, and creating an index to the video sequence based onidentities of the first camera and the second camera.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a facility including a plurality of video cameras,according to various embodiments of the invention.

FIG. 2 is a block diagram of a video tracking system, according tovarious embodiments of the invention.

FIG. 3 illustrates an example of a user interface, according to variousembodiment˜of the invention.

FIGS. 4A-4C illustrate several ways in which a video display may bedivided into selectable regions, according to various embodiments of theinvention.

FIG. 5 illustrates alternative embodiments of a video trackinginterface.

FIG. 6 illustrates methods of presenting video data in a user interface,according to various embodiments of the invention.

FIG. 7 illustrates methods of tracking a target according to variousembodiments of the invention.

FIG. 8 illustrates methods of stitching video data, according to variousembodiments of the invention.

DETAILED DESCRIPTION

Tracking of a target between multiple video streams may be facilitatedby a user interface configured to use relationships between the view ofone camera and the views of other cameras according to a cameratopology. The camera topology is based on spatial relationships betweencameras relative to a view point of one of the cameras. This is incontrast to a spatial relationship that is defined relative to anexternal coordinate or orientation system.

Various embodiments of the invention include a user interface comprisingspatially intuitive camera selection controls for navigating betweencameras and the video data generated by these cameras. These cameraselection controls are arranged within the user interface according tothe camera topology. For example, as further described herein, if atarget moves to the right out of the field of view of a first camera, acamera selection control disposed to the right of other camera selectioncontrols can be used to select a second camera whose field of view is tothe right of the field of view of the first camera.

FIG. 1 illustrates an exemplary Facility generally designated 100 andincluding a plurality of Cameras 105. These Cameras 105 are individuallydesignated 105A-105H. Each of Cameras 105 is associated with at leastone field of view, illustrated using Dashed Lines 110 in FIG. L A memberof Cameras 105 that is remotely movable may be associated with more thanone field of view and, thus, more than one camera topology. Facility 100includes Walls 115 and other Objects 120 that may define corridors, openareas, and the like. Objects 120 may include movable objects such asvehicles, curtains, inventory, temporary structures, signs, chattel, orthe like. Walls 115 and Objects 120 may obscure the fields of view ofCameras 105. The Facility 100 illustrated in FIG. I is for illustrativepurposes, one of ordinary skill in the art would understand that thevarious embodiments of the invention described herein may be adapted toa wide variety of facilities including airports, schools, governmentinstitutions, ports, ships, warehouses, military installations, powerplants, transportation infrastructure, cities, and the like.

Some of the topological relationships between Cameras 105 in Facility100 can be described as follows. Camera 105A is disposed to the left ofCamera 105B and Camera 105B is disposed to the right of Camera 105A.These orientations are determined by facing in the directions of Cameras105A and 105B, respectively, and observing the right or left location ofthe other camera. Camera 105C is disposed to the upper left of Camera105A, and Camera 105A is disposed to the upper right of Camera 105C.Further, Camera 105F is disposed to the center of Camera 105B, andCamera 105B is disposed to the upper right of Camera 105F

In some instances, the fields of view of different members of Cameras105 overlap. For example, the fields of view of Cameras 105A and 105Boverlap. Some parts of Facility 100 may not be included in the field ofview of any of Cameras 105. For example, a Corridor 125 is disposedbetween fields of view of a Camera 10FD and a Camera 105E.

Cameras 105 may include a variety of different camera types, havingdifferent focal lengths, resolutions, spectral sensitivity, movementcapability, and output type. For example, members of Cameras 105 may beconfigured to generate analog video signals, or digital video data in avariety of formats. Members of Cameras 105 may be Pan-Tilt-Zoom (PTZ)cameras that are configured to be remotely controlled. Members ofCameras 105 may be portable and may include Global Positioning System(OPS).

FIG. 2 is a block diagram of a Video Tracking System 200, according tovarious embodiments of the invention. Video Tracking System 200 includeslogic and/or hardware configured for tracking a target using video datafrom Cameras 105. The logic may include hardware, firmware, and/orsoftware stored on a computer readable medium. The hardware may includeintegrated circuits, a display device, cameras, data storage, and thelike. For example, some embodiments include an integrated circuitconfigured to execute software to generate a user interface on a displaydevice. The user interface is configured to display video stored in astorage device. As is further described herein, the user interfacetypically includes one or more user controls disposed within the userinterface according to a camera topology.

In the embodiments illustrated by FIG. 2, Video Tracking System 200includes a User Interface Generation Logic 205 configured to generate auser interface. User Interface Generation Logic 205 may includehardware, firmware, and/or software stored on a computer readablemedium. For example, in some embodiments, User Interlace GenerationLogic 205 includes computing instructions configured to be executed by aProcessor 210. Processor 210 may include a microprocessor, centralprocessing unit (CPU), and/or the like. In some embodiments, Processor210 is part of a personal computer or server.

Briefly, the output of User Interface Generation Logic 205 is configuredto be presented as a user interface to a user via a Display Device 215,and may include images (e.g., video or still pictures), metadata, videocontrol signals, links, and/or the like. The user interface isoptionally presented in a graphical interface such as Microsoft Windows™or OS 10 from Apple Computer. The user interface may include bothfeatures visible to the user and underlying functionality. Displaydevice 215 may include one or more computer monitors, a liquid crystaldisplay, a plasma display, and/or the like.

The user interface generated using User Interface Generation Logic 205is configured to present video to the user. This video can be real-time(live) video or recorded video. For example, in some embodiments, thevideo presented includes live video received through members of Cameras105. In some embodiments, the video presented includes video previouslystored in a Video Data Storage 220. Video Data Storage 220 may includeoptical storage, a hard drive, static memory, random access memory,and/or other computer readable medium.

Video data received from Cameras 105 may be received through an optionalVideo Input Interface 225. Video Input Interface 225 is configured toreceive video in a variety of formats and to convert the received videoto a common format For example, Video Input Interface 225 may beconfigured to receive video in analog formats such as RGB(red-green-blue) or YPbPr (luma-blue-red), or digital formats such asMPEG 1 (ISO: Moving Pictures Experts Group), MPEG2, MPEG4, H.261(International Telecommunication Union), H.263, H.264, MJPEG, WMV(International Telecommunication Union), etc. and to convert thereceived video to MJPEG. Video Input Interface 225 optionally includesRCA, S-Video, DIN, DB25, USB, or Ethernet connectors configured tocouple to different types of Cameras 105.

Video Input Interface 225 is optionally further configured to add timeand source data to the received video data, and to store the receivedvideo data, with time and source data, in Video Data Storage 220. Forexample, the data stored in Video Data Storage 220 may include a datavalue indicating which of Cameras 105 the video data came from, as wellas one or more time markers configured for associating different partsof the video data with a time of day. As is described further herein,the time markers may be used to temporally synchronize video datareceived from different members of Cameras 105.

Video Tracking System 200 further includes a Camera Topology Storage 230configured to store the topological relationships between Cameras 105.As described elsewhere herein, these topological relationships aredefined based on the point of view of each camera. These relationshipsare optionally established using a click-and-drag setup utility or maybe automatically determined based on a mapping of camera locations andorientations.

FIG. 3 illustrates an example of a user interface, Video TrackingInterface 300, as may be generated using User Interface Generation Logic205, according to various embodiments of the invention. In theseembodiments, Video Tracking Interface 300 includes a Main Video Display305 and a plurality of User Controls 310, individually designated310A-310B. Main Video Display 305 is a part of Video Tracking Interface300 configured to present video data to a user. The video data presentedin Main Video Display 305 may be real-time video or video previouslystored in Video Data Storage 220. The user may observe a target, e.g., aperson or a vehicle, in the Main Video Display 305.

Each of User Controls 310 can be associated with a different member ofCameras 105, and is configured for a user to select between video datagenerated using these different members of Cameras 105. For example, ifthe user activates User Control 310A, video generated using a member ofCameras 105 that is associated with User Control 310A will be presentedin Main Video Display 305. If the user activates User Control 310B,video generated using a member of Cameras 105 that is associated withUser Control 310B will be presented in Main Video Display 305.

In various embodiments, the association of User Controls 310 withmembers of Cameras 105 based on camera topology results in an intuitiveinterface for tracking of targets between video camera. For example, ifthe target moves to the right out of the field of view presented in MainVideo Display 305, then User Control 31 OB can be used to select themember of Cameras 105 whose field of view the target is most likely tobe found next. As the target moves from camera to camera, a user canselect those members of User Controls 310 adjacent to the edge of MainVideo Display 305 at which the target disappeared.

The associations between User Controls 310 and different members ofCameras 105 are managed by Video Data Selection Logic 235 according tothe topological relationships between Cameras 105 and according to whichvideo data is currently being presented in Main Video Display 305. Forexample, if video data generated using Camera 105B (see FIG. 1), iscurrently being displayed in Main Video Display 305, then thetopological relationships will include that Cameras 105G and 1050 are tothe right, with Camera 105D being more the upper right and Camera 105Gmore to the lower right. Likewise, Camera 105A is to the lower left,Camera 105F is directly ahead, and Camera 105C is to the upper right.

Video Data Selection Logic 235 is configured to use these relationshipsto associate members of User Controls 310 to specific members of Cameras105 such that the spatial distribution of User Controls 310 within VideoTracking Interface 300 is mapped to the topological relationshipsbetween cameras. Thus, since Camera 105A is to the left of Camera 105B,when video generated using Camera 105B is displayed in Main VideoDisplay 305, User Control 310D will be associated with Camera 105A.Likewise, User Control 310G (and/or 310A) will be associated with Camera105F, User Control 310B will be associated with Camera 105D, UserControl 310H will be associated with Camera 105C, and User Control 310Ewill be associated with Camera 1050.

Some of User Controls 310 may not be associated with members of Cameras105. For example, when video generated using Camera 105B is displayed inMain Video Display 305, User Control 310F is not associated with amember of Cameras 105 because there is no camera to the lower left fromthe point of view of Camera 105B. User Control 310F is optionally,grayed out, absent, or otherwise configured to indicate the lack ofassociation with a member of Cameras 105. For example, in someembodiments, the appearance of members of User Controls 310 is dependenton whether or not they are associated with a source of video data. Insome embodiments, the sizes, shapes and/or positions of User Controls310A-310H may change based on the availability of Cameras 105 in thecurrent camera topology. For example, if there is only one of Cameras105 to the left, then User Control 310D may extend along the entire leftedge of Main Video Display 305. More than one member of User Controls310 may be associated with the same member of Cameras 105.

Video Data Selection Logic 235 is configured to change the video datapresented in Main Video Display 305 if a user activates one of UserControls 310. The video data presented is that that is or was generatedusing the member of Cameras 105 that is associated with the activatedmember of User Controls 310. For example, if User Control 3100 isassociated with Camera 105F, then activation of User Control 3100 willresult in presentation of video data generated using Camera 105F in MainVideo Display 305.

Video Data Selection logic 235 is configured to change to theassociations between User Controls and members of Cameras 105 when amember of User Controls 310 is activated. For example, if a member ofUser Controls 310 associated with Camera 105F is activated, in additionto the presentation of different video data in Main Video Display 305,members of User Controls 310 are re-associated with members of Cameras105 according to the camera topology of Camera 105F. Specifically, sinceCamera 105A and Camera 105B are to the right and upper right,respectively, of Camera 105F, Camera 105A may be associated with UserControl 310B and Camera 105B may be associated with User Control 31 OH.Likewise, Camera 105D may be associated with User Control 3100, andCamera 105C may be associated with User Control 310D. Each time thesource of the data presented in Main Video Display 305 is changed, oneor more of the associations between members of User Controls 310 andCameras 105 may automatically be changed.

While the embodiments of Video Tracking Interface 300 illustratedinclude eight User Controls. 310, (310A-310H), alternative embodimentsmay include different numbers of User Controls 310. For example, someembodiments include four User Controls 310, one on each side, one belowand one above Main Video Display 305. Some embodiments include more thaneight User Controls 310, for example, in situations where a target maymove up or down (e.g., using stairs), as well as directly toward or awayfrom a member of Cameras 105, additional user controls may be used totrack a target moving in these directions. In one embodiment, anadditional member of User Controls 310 is dedicated for association withan “Exit Door.” In one embodiment, User control 310A is configured totrack a target moving up and User Control 310C is used to track a targetmoving down. A ninth and tenth member of User Controls 310 are used totrack a target moving directly away and toward the camera. Theseadditional User Controls 310 are illustrated, for example, in FIG. 5below.

User Controls 310 can be selectable (clickable or touch sensitive)images shown on Display Device 215 or may include physical buttonsdisposed around Display Device 215. When User Controls 310 are shown onDisplay Device 215, they are optionally incorporated within Main VideoDisplay 305. In these embodiments, Main Video Display 305 may be dividedup into parts such that clicking on these parts results in theactivation of User Controls 310.

FIGS. 4A-4C illustrates several ways in which Main Video Display 305 maybe divided into selectable regions, according to various embodiments ofthe invention. In FIG. 4A four regions are defined. In FIG. 4B eightregions are defined. In FIG. 4C nine regions are defined, including aCentral Region 410. The regions labeled 310A-310H operate as UserControls 310. Central Region 410 may operate as an additional member ofUser Controls 310, and may be associated with a camera topology for adirection directly away in the viewpoint of the camera whose video datais currently being presented in Main Video Display 305. As describedelsewhere herein, additional User Controls 310, associated with cameratopologies for directions directly away or directly toward the viewpointof the camera may be disposed outside Main Video Display 305.

In alternative embodiments, User Controls 310 are disposed in one ormore groups beside Main Video Display 305 rather than surrounding MainVideo Display 305. In these embodiments, User Controls 310A-310H maystill be spatially arranged within the one or more groups so as toindicate their functionality and relationship to camera topology.

In some embodiments, a member of User Controls 310 is associated withmore than one member of Cameras 105. For example, when data generatedusing Camera 105F is being displayed in Main Video Display 305, UserControl 310D may be associated with both Camera 105C and Camera 105D.When a member of User Controls 310 having multiple associations isselected by a user, the user may be presented with a list of alternativemembers of Cameras 105 from which to choose. For example, in someembodiments, the user is presented with a group of thumbnails showingvideo data generated by each of the members of Cameras 105 that isassociated with the selected member of User Controls 310. In someembodiments, the list of alternative members of Cameras 105 is orderedbased on a prediction of which member of Cameras 105 is most likely tobe wanted by a user. For example, it may be determined by Video DataSelection Logic 235 that when a target moves to the left from the fieldof view of Camera 105F there is a 33 percent chance that the user willbe observed next in the field of view of Camera 105D and a 66 percentchance that the user will be observed next in the field of view ofCamera 105C. Presentation of alternative members of Cameras 105 may beuseful, particularly when an environment changes. For example, movementof Objects 120 may block one member of Cameras 105 and, thus, makeanother member preferable for viewing.

In some embodiments, each of User Controls 310 includes a thumbnail ofvideo data generated using the respective associated member of Cameras105. For example, when Video Tracking Interface 300 is displayed on acomputer monitor, each of User Controls 31 OA-31 OH may be representedby a video thumbnail. In some embodiments, Video Tracking Interface 300is configured such that moving a cursor over one of User Controls310A-310H will result in displaying of video data generated using theassociated member of Cameras 105. This video data may be displayed as athumbnail or in a region of Video Tracking Interface 300 other than MainVideo Display 305.

Returning to FIG. 2, a Video Synchronization Clock 240 is configured tosynchronize video data generated using different members of Cameras 105.As discussed elsewhere herein, the video data stored in Video DataStorage 220 includes information identifying which of Cameras 105 thevideo data was generated by and time information relating the video datato a time of day, or similar time reference. The time information isused by Video Data Selection Logic 235 and Video Synchronization Clock240 to move between video data from different sources in a timesynchronized manner. For example, if a user is viewing data generatedusing Camera 105G in Main Video Display 305 and then uses User Control310B to select data generated using Camera 1050, then data generatedusing Camera 1050 will be presented in Main Video Display 305 and thepart of the data generated using Camera 105 D presented will be thatpart that is adjacent in time to the video data just viewed from Camera105G.

In a more specific example, if a user begins watching video data fromCamera 105G that was recorded at 1:00, watches for 5 minutes until 1:05,and then switches to watching video data from Camera 105D, then the datapresented in Main Video Display 305 will be that data recorded at 1:05using Camera 105D. Likewise, if a user begins watching video data fromCamera 1050 that was recorded at 1:00, watches backwards in time(showing the video data in reverse) until 12:55, and then switches towatching video data from Camera 105D, then the data presented in MainVideo Display 305 will be that data recorded at 12:55 using Camera 105D.Typically, because the video data from Camera 105G was being shown inreverse and the data from Camera 105D will also be shown backwards intime, although the user may reverse direction of the viewing of videodata.

The synchronization between video data optionally includes an offsetselected to approximate the time a target would be expected to travelbetween members of Cameras 105. For example, if a typical travel timebetween the fields of view of cameras 1050 and 105D is 15 seconds, thena 15 second offset is optionally automatically used when synchronizingvideo data from Camera 1050 and Camera 105D. If a user is viewing videodata from Camera 1050 in Main Video Display 305, and uses User Control310B to select video data generated by Camera 105D starts by viewingvideo data generated using Camera 105D, then the video from Camera 105Dwill be offset by 15 seconds. This offset may enable the user to see thetarget immediately in the field of view of Camera 1050 without having towait for the target to travel between fields of view.

A synchronization offset may also be used when presenting video datawithin thumbnails. For example, if User Control 310B includes athumbnail of video data generated using Camera 105D, while video datafrom Camera 1050 is displayed in Main Video Display 305, then the videoshown in the thumbnail may be offset by 15 seconds from the videopresented in Main Video Display 305.

Referring again to FIG. 2, Video Tracking System 200 optionally includesa Stitching Logic 245 configured to form a video sequence out of videodata generated using more than one member of Cameras 105. The videosequence includes video data from different members of Cameras 105 in achronological order. The video sequence may be automatically generatedas a target is tracked using Video Tracking System 200. For example, asa user uses User Controls 310 to select which video data to present inMain Video Display 305, the presented video data may be automaticallyincluded in a stitched video sequence. In some embodiments, the stitchedsequence includes a copy of the video data configured in a sequentialvideo file. In some embodiments, the stitched sequence includes a seriesof pointers, e.g., file names and time references, to other files andthe video data included of the stitched sequence need not be stored as asingle unit.

A video sequence may include video presented forward and/or backwards intime, and may include time reversals. For example, a video sequence mayinclude video data generated using Camera 105A viewed backwards in timestitched to video data generated using Camera 105D. The video datagenerated using Camera 105D, as included in the video sequence, mayinclude a time reversal, e.g., where a user started watching the videodata backward in time and changed the temporal direction of the videodata presentation to watch the same video data forwards in time. Thesequence may next include video data generated using Camera 105E viewedforward in time. Such video sequences may be useful for tracking a firsttarget back in time until some event occurs and then tracking a secondtarget forward in time following the event.

In some embodiments, Stitching Logic 245 is configured to automaticallyindex the stitched video sequence using camera identificationinformation and/or thumbnail stills of the video sequence. This indexmay be used to navigate forward and back in time within the video data.For example, if a user is tracking a target using recorded video dataand the target travels an unexpected route such that the target does notappear in video data generated using a member of Cameras 105 selectedusing User Controls 310, then the user can use the stitched sequence tobacktrack and re-acquire the target Thus, Stitching Logic 245 isoptionally configured for editing, correcting and reorganizing videosequences.

An optional Report Generator 250 is configured to generate a reportusing data generated by Video Tracking System 200. For example, in someembodiments, Report Generator 250 is configured for generating a listingof areas through which a target travels based on which members ofCameras 105 are used to track the target. In some embodiments, ReportGenerator 250 is configured to automatically include a series ofthumbnails in an electronic document. These thumbnails may be indexes toa video sequence generated using Stitching Logic 245 and may includelinks to the video sequence.

In some embodiments, Video Tracking System 200 includes a GPS (globalpositioning system) Input 255 configured to receive location andoptional orientation information from a camera, such as a member ofCameras 105. For example, GPS Input 255 may be configured to receiveinformation regarding the location of a portable member of Cameras 105.This information may be generated using a GPS system, a localpositioning system, a radio triangulation system, an inertia measurementdevice, and/or other position sensing system. GPS Input 255 is notlimited to receiving position information from positioning systems thatare global. The optional orientation information is configured fordetermining a direction in which the camera is pointed. Video DataSelection Logic 235 may use the location and orientation information todetermine topological relationships between a portable member of Cameras105 and fixed members of Cameras 105. These topological relationshipsare optionally determined in real-time and updated as the portablecamera is moved. The portable camera may be included in a cellulartelephone, personal digital assistant, communication device, or thelike. OPS Input 255 may include a wireless receiver, a universal serialbus, a powerline communication system, a network adaptor, and/or thelike.

FIG. 5 illustrates alternative embodiments of Video Tracking Interface300 including an interactive Map 510 of Facility 100, a stitchedsequence represented by Sequence Thumbnails 520A-520D, a PreviewThumbnail 525, and various additional user controls. The variousfeatures illustrated in FIG. 5 may be included in alternativecombinations and/or arrangements in alternative embodiments of theinvention. In the embodiments illustrated, some members of User Controls310A-310H are grayed out because there are currently no members ofCameras 105 associated with these controls. For example, from the pointof view of Camera 105A there is no camera to associate with User Control310D.

Map 510 is configured to show a user which of Cameras 105 is currentlyactive in Main Video Display 305. For example, in the embodimentillustrated, a Circle 530 is used to indicate that data from Camera 105Ais currently being presented in Main Video Display 305. Circle 530 isoptionally brightly colored or flashing. Circle 530 may move to mark adifferent member of Cameras 105 when a user selects video data from thedifferent member of Cameras 105. Map 510 optionally further includes aPath Indicator 535 representative of a path taken by a target throughFacility 100.

Sequence Thumbnails 520A-520D are an index to a video sequence generatedas a target travels through Facility 100. This video sequence isoptionally generated automatically in response to a user's operation ofUser Controls 310, as described elsewhere herein. Each of SequenceThumbnails 520A-520D includes a still image and is associated with atleast one time point and a member of Cameras 105. The video sequence canbe navigated by selecting one of the Sequence Thumbnails 520A-520D. Parexample, clicking on Sequence Thumbnail 520B may cause a part of thevideo sequence generated using Camera 1050 at time 1:44:50 to bepresented in Main Video Display 305. The video sequence represented bySequence Thumbnails 520A-520D may also be navigated and edited usingSequence Controls 540.

Sequence Thumbnails 520A-520C may be generated automatically each timedata from a different member of Cameras 105 is presented in Main VideoDisplay 305 or by selecting a Create Thumbnail Control 545. For example,if a user observes a particular event to be documented, the user can useCreate Thumbnail Control 545 to index the video sequence at the time ofthat event. Thumbnails can be created for both real-time and previouslyrecorded video.

Preview Thumbnail 525 may be used to view video generated by a member ofCameras 105. For example, in some embodiments, moving a cursor over amember of User Controls 310 will cause video generated by an associatedmember of Cameras 105 to be presented in Preview Thumbnail 525. Someembodiments include a plurality of Preview Thumbnail 525.

Video Tracking Interface 300 optionally further includes additional UserControls 550A and 550B, configured for tracking a target up or downrelative to the viewpoint of a member of Cameras 105. For example, asdiscussed elsewhere herein, these controls may be used for tracking atarget up and down stairs.

Video Tracking Interface 300 optionally further includes Camera MovementControls 555 configured to operate a movable (e.g., PTZ) camera.Typically, Camera Movement Controls 555 are automatically associatedwith whichever member of Cameras 105 is currently being used to generatevideo data presented in Main Video Display 305.

Video Tracking Interface 300 optionally includes a Report GenerationControl 560 configured to execute a report generation utility. Thereport generation utility may automatically include video sequences,thumbnails, or other data generated using Video Tracking Interface 300.For example, in some embodiments report generation utility includes atemplate to which a user can add notes to thumbnails.

Video Tracking Interface 300 optionally includes a Camera Locate Control565. Camera Locate Control 565 is configured to determine a location ofa portable member of Cameras 105, and optionally to display thedetermined location on Map 510. Similar features may be used to locateother objects and/or security personnel.

Video Tracking Interface 300 optionally includes an Alert ActivationControl 570. In various embodiments, activating Alert Activation Control570 triggers alarms, logging and/or various actions according topre-established procedures. These procedures may be defined by a set ofrules and be performed by a rule engine (not shown) included in VideoTracking System 200. The rules may define a workflow to be performed inresponse to an alert, automatic indexing of the video data beingobserved when Alert Activation Control 570 is selected, automaticgeneration of a report or opening of a report template, etc. Forexample, if a user observes tailgating in a video, the user can selectAlert Activation Control 570 to index the video data and enter notesregarding the event in a report template. These notes may includeidentities of the parties involved in the tailgating, where it occurred,how many people were involved, what action was taken, etc. In oneembodiment, the report will automatically include video data recorded 5seconds before and 5 seconds after the event

The various features illustrated in FIG. 5 are typically supported byunderlying logic and/or hardware configured to perform associatedoperations. This underlying logic may be part of Video Tracking System200. For example, Camera Locate Control 365 may be supported by OPSInput 255 and logic configured to map a location to Map 510. CameraMovement Controls 555 may be supported by logic that automaticallyassociates the controls with one of Camera 105 responsive to UserControls 310A-310H.

FIG. 6 illustrates methods of presenting video data in a user interface,according to various embodiments of the invention. In this methodtopographical relationships are identified between a first member ofCameras 105 and other members of Cameras 105, video data from the firstmember of Cameras 105 is optionally converted into a required format andpresented in Video Tracking Interface 300, members of User Controls 310are associated with the other members of Cameras 105, one of the UserControls 310 is selected, video data from a second member of Cameras 105is presented in Video Tracking Interface 300, and the User Controls 310are again associated with members of Cameras 105. Optionally, video datafrom the first member and second member of Cameras 105 is stitchedtogether to generate a video sequence.

More specifically, in an Identify First Camera Step 605, a first memberof Cameras 105 is identified. This camera may be, for example, Camera105A as illustrated in FIG. 5.

In an Identify First Relationship Step 610, a first topologicalrelationship between the member of Cameras 105 identified in IdentifyFirst Camera Step 605 and a second member of Cameras 105 is identified.For example, if the second member of Cameras 105 is Camera 105B, thetopological relationship may include that the second member is to theright of the first member. This first topological relationship isrelative to the viewpoint of the first member of Cameras 105, e.g.,Camera 105A.

In an Identify Second Relationship Step 615, a second topologicalrelationship between the first member of Cameras 105 and a third memberof Cameras 105 is identified. For example, if the third member ofCameras 105 is Camera 105F, then the topological relationship mayinclude that the third member is to the left of the first member. Thesecond topological relationship is relative to the viewpoint of thefirst member of Cameras 105.

In an optional Convert Video Data Step 620, video data received from thefirst member of Cameras 105 is converted into a desired format. Thisconversion may be performed using Video Input Interface 225, and mayinclude associating camera identification information and time referenceinformation with the video data. As is described elsewhere herein, VideoInput Interface 225 may be configured to convert video data from avariety of formats to a common format.

In a Present Video Data Step 625, video data generated using the firstmember of Cameras 105, identified in Identify First Camera Step 605, ispresented in Main Video Display 305. This video data may be real-timevideo data, or previously recorded video data stored in Video DataStorage 220.

In an Associate Second Camera Step 630, a second member of Cameras 105is associated with a first member of User Controls 310. For example, ifthe first member of Cameras 105 is Camera 105A, then Camera 105B may beassociated with User Control 310B. The association is typically based onthe topological relationship between the first member of Cameras 105 andthe second member of Cameras 105, as identified in Identify FirstRelationship Step 610. For example, if the topological relationshipincludes that the second member is to the right of the first member,then a member of User Controls 310 to the right (e.g., User Control 31OB) is associated with the second member of Cameras 105.

In all Associate Third Camera Step 635, a third member of Cameras 105 isassociated with a second member of User Controls 310. This associationis typically based on the topological relationship between the firstmember of Cameras 105 and the third member of Cameras 105 as identifiedin Identify Second Relationship Step 615. The first and second membersof User Controls 310 are optionally presented in Video TrackingInterface 300 in a spatial arrangement representative of the firsttopological relationship and the second topological relationship.

In a Receive Selection Step 640, a user selection of one of UserControls 310 is received. This selection may be the result of a buttonbeing pressed, a click in Video Tracking Interface 300, or the like.Sometimes, the selection is made as the result of a target leaving afield of view associated with the first member of Cameras 105. For thesake of illustration, it is assumed that the selected member of UserControls 31 O is the second member of User Controls 310.

In a Present Video Data Step 645, video data generated using the memberof Cameras 105 selected in Receive Selection Step 640 is presented inMain Video Display 305. For example, if the selected member of UserControls 310 is the second member of User Controls, e.g., User Control310B, then video data generated using the second member of Cameras 105is presented. This video data may be real-time video data, or previouslyrecorded video data stored in Video Data Storage 220.

In an Associate Controls Step 650, the first member of User Controls 310and the second member of User Controls 310 are again associated withmembers of Cameras 105. These associations are based on topologicalrelationships relative to a viewpoint of the member of Cameras 105selected in Receive Selection Step 640. For example, if the member ofCameras 105 selected in Receive Selection Step 640 is Camera 105B, thenUser Control 310D may be associated with Camera 105A. User Control 310Emay be associated with Camera 1050, etcetera.

In an optional Stitch Video Step 655, video data presented in PresentVideo Data Step 625 and video data presented in Present Video Data Step645 are stitched together to form a video sequence. Stitch Video Step655 is optionally performed automatically in response to ReceiveSelection Step 640. Further details of stitching video data to form avideo sequence are discussed elsewhere herein. Stitch Video Step 655optionally includes indexing of the video sequence and/or generation ofthumbnails configured for navigating the video sequence. TheseThumbnails may be presented in Video Tracking Interface 300.

FIG. 7 illustrates a method of tracking a target according to variousembodiments of the invention. In this method a target is first locatedin video data generated using a first member of Cameras 105. When thetarget leaves the field of view of the first member of Cameras 105, asecond member of Cameras 105 is selected using User Controls 310 and thetarget is again observed using video data generated using the secondmember of Cameras 105. This process may be repeated using a third memberof Cameras 105.

The methods illustrated by FIG. 7 may include live video, recorded videopresented to a user in real-time, or recorded video presented at a ratefaster than real time, e.g. in an accelerated mode. Further, the videomay be viewed forward or backwards in time. In some embodiments, thedirection of video presentation is different in different steps of FIG.7. For example, the target observed may include more than one person orobject. A first person may be observed in real-time video, tracked backin time until the first person meets a second person, and then thesecond person is tracked forward in time until the second person isviewed in real-time.

Typically, the video data from different members of Cameras 105 ispresented to a user for viewing in a synchronized manner. As such, theuser can move from one camera to another while viewing video data thatis approximately continuous in time. As video data from differentsources is viewed, the video data is optionally automatically stitchedtogether to form a video sequence.

Specifically, in a Locate Target Step 705, a target to be tracked islocated in video data presented in Main Video Display 305. This video istypically generated using a first member of Cameras 105, for example,Camera 105E as illustrated in PIG. 5.

In an Observe Target Step 710 the target is observed using VideoTracking Interface 300. Often the target will eventually leave the fieldof view of the first member of Cameras 105.

When the target leaves the field of view of the first member, a userselects a second member of Cameras 105 using one of User Controls 310,in a Select Camera Step 715. In some embodiments, the user can make thisselection by choosing the member of User Controls 310 closest to theedge of Main Video Display 305 through which the target left the fieldof view of the first member of Cameras 105. For example, if the targetleaves the field of view through the upper left edge, the user mayselect User Control 310G.

In a View Video Data Step 720, video data generated using the member ofCameras 105 selected in Select Camera Step 715, i.e., the second member,is viewed in Main Video Display 305.

In an optional Stitch Video Data Step 725, the viewed video datagenerated using the first and second members of Cameras 105 is stitchedtogether to form a video sequence.

In an optional Observe Target Step 730 the target is observed in thevideo data generated using the second member of Cameras t 05. Often thetarget will eventually leave the field of view of the second member ofCameras 105. When the target leaves the field of view of the secondmember of Cameras 105, the user may select a third member of Cameras 105using one of User Controls 310, in a Select Camera Step 735. Thisselection can be made in manners similar to those described above withrespect to Select Camera Step 715.

In an optional View Video Data Step 740, video data generated using thethird member of Cameras 105 is viewed in Main Video Display 305.

When video data is viewed backwards in time, the target may be observedentering fields of view rather than leaving fields of view. Further, asdiscussed elsewhere herein, the various video data viewed in the stepsillustrated in FIG. 7 may be temporally synchronized. Thissynchronization may include use of a synchronization offset based on anestimated time for a target to travel between camera fields of view.

FIG. 8 illustrates methods of stitching video data, according to variousembodiments of the invention. In these methods, video data is selectedby a user, for example using User Controls 310, and automaticallystitched together to form a video sequence. The video sequence istypically chronological in time and may be indexed based on sources ofthe video data.

Specifically, in a Receive First Selection Step 805, a user selection offirst video data, generated using a first member of Cameras 105, isreceived. Typically, the first video data includes a first timereference and camera identifying information. The user selection isoptionally made using a member of User Controls 310.

In a Receive Second Selection Step 810, a user selection of second videodata, generated using a second member of Cameras 105, is received.Typically, the second video data includes a second time reference andcamera identifying information. The user selection is optionally madeusing a member of User Controls 310.

In a Stitch Selections Step 815, the first video data and the secondvideo data are stitched together to form a chronological video sequence.Stitch Selections Step 815 is optionally performed automatically inresponse to the selection received in Receive Second Selection Step 810.Stitch Selections Step 815 optionally makes use of a temporal off set,as described elsewhere herein. Thus, the video sequence can either betemporally contiguous or include a temporal offset between video data.The video sequence may be stored as a single set of video data in VideoData Storage 220, or may be stored as a set of pointers to video data.The video sequence may include a timer reversal.

In an optional Index Sequence Step 820, the video sequence formed inStitch Selections Step 815 is indexed. This index may be based on thefirst or second time references, notes provided by a user, thumbnailscreated by the user using Create Thumbnail Control 545, or theidentities of the first and second members of Cameras 105. In someembodiments, the index is associated with thumbnails of the video dataincluded in the video sequence. Stitch Selections Step 815 and IndexSequence Step 820 are optionally performed using Stitching Logic 245.Index Sequence Step 820 is optionally automatically performed inresponse to a selection of one of User Controls 310.

Several embodiments are specifically illustrated and/or describedherein. However, it will be appreciated that modifications andvariations are covered by the above teachings and within the scope ofthe appended claims without departing from the spirit and intended scopethereof. For example, Video Tracking System 200 may include an automaticmode in which a target is automatically tracked from the viewpoint ofone of Cameras 105 to another of Cameras 105 based on targetidentification. For example, target identification technology disclosedin U.S. patent application Ser. No. 11/229,091, filed Sep. 16, 2005 andentitled “Robust Perceptual Color Identification” may be used toidentify a target. Video Tracking System 200 may further be configuredto store a catalog of targets and/or index video data including specifictargets. Video Tracking System 200 may be configured to work with audioas well as video data Various embodiments of the invention includecomputing instructions stored on computer readable media and configuredfor performing methods of the invention.

The embodiments discussed herein are illustrative of the presentinvention. As these embodiments of the present invention are describedwith reference to illustrations, various modifications or adaptations ofthe methods and or specific structures described may become apparent tothose skilled in the art All such modifications, adaptations, orvariations that rely upon the teachings of the present invention, andthrough which these teachings have advanced the art, are considered tobe within the spirit and scope of the present invention. Hence, thesedescriptions and drawings should not be considered in a limiting sense,as it is understood that the present invention is in no way limited toonly the embodiments illustrated.

What is claimed is:
 1. A method, comprising: under control of one ormore processors, receiving first and second video data from first andsecond cameras distributed over a monitored area; presenting a videotracking interface, displaying, on the video tracking interface, firstand second user controls corresponding to the first and second cameras,positioning the first and second user controls around a main videodisplay; receiving a selection of the first user control; in response toselection of the first user control, displaying the first video datafrom the first camera in the main video display; in response to a cursorbeing moved over the second user control, presenting the second videodata from the second camera in addition to the first video data.
 2. Themethod of claim 1, wherein the second video data is displayed as apreview thumbnail or in a region of the video tracking interface otherthan the main video display.
 3. The method of claim 1, furthercomprising storing a video sequence that includes at least one of thefirst and second video data.
 4. The method of claim 3, furthercomprising presenting, on the video tracking interface, a timeline thatcorresponds to the video sequence.
 5. The method of claim 3, wherein thevideo tracking interface presents a set of sequence thumbnails that areindexed to points along the video sequence, the sequence thumbnailscorrespond to points in time at which video data from different camerasis presented in the video sequence.
 6. The method of claim 5, wherein,when one of the sequence thumbnails is selected, the video trackinginterface presents a corresponding part of the video sequence in themain video display.
 7. The method of claim 1, further comprisinggenerating a report that includes a video sequence and thumbnail.
 8. Themethod of claim 1, further comprising presenting an interactive map ofthe monitored area on the video tracking interface, the interactive mapillustrating at least one of objects or walls within the monitored areaand illustrating the first and second cameras relative to the at leastone of objects or walls, the interactive map indicating first or secondcamera that is currently active in the video display.
 9. The method ofclaim 1, further comprising indicating, on the interactive map, thefirst or second camera that is active by at least one of a circle, coloror flash.
 10. The method of claim 1, further comprising associating agroup of cameras with the second user control; and in response toselection of the second user control, presenting a list of the group ofcameras from which to choose.
 11. The method of claim 1, furthercomprising providing an alert activation control to at least one oftrigger alarms, log events or performs actions according topre-established procedures, the procedures defined by a set of rulesthat define a workflow to be performed in response to an alert.
 12. Amethod comprising: utilizing one or more processors for, receiving firstand second video data from first and second cameras distributed over amonitored area; storing a video sequence that includes at least one ofthe first and second video data; presenting a video tracking interface,displaying, on the video tracking interface, first and second usercontrols corresponding to the first and second cameras, in response toselection of the first user control, displaying, on a main video displayof the video tracking interface, first video data from the first camera,the first and second user controls positioned around the main videodisplay, presenting, on the video tracking interface, a timeline thatcorresponds to the video sequence, and presenting, on the video trackinginterface, a set of sequence thumbnails that are indexed to points alongthe video sequence.
 13. The method of claim 12, wherein the videosequence is generated automatically in response to user operation of thefirst and second user controls.
 14. The method of claim 12, furthercomprising automatically indexing the first and second video data,automatically generating a report, and enabling a user to enter notesinto the report regarding an event.
 15. The method of claim 12, furthercomprising associating a group of cameras with the second user control,in response to selection of the second user control, presenting a listof the group of cameras from which to choose, the list of the group ofcameras ordered based on a prediction of which member of the cameras ispreferred.
 16. The method of claim 12, further comprising tracking atarget from recorded video data to real-time video data.
 17. The methodof claim 12, further comprising tracking a target back in time throughthe video sequence.
 18. The method of claim 12, further comprisingstoring an estimated travel time between the first and second cameras,and utilizing a time offset based on the estimated travel time.
 19. Themethod of claim 12, further comprising automatically stitch portions ofthe first and second video data into the video sequence based onnavigation between the first and second cameras as performed using thefirst and second user controls.